Designing Illustrated Texts: How Language Production Is Influenced By Graphics Generation
نویسندگان
چکیده
Multimodal interfaces combining, e.g., natural language and graphics take advantage of both the individual strength of each communication mode and the fact that several modes can be employed in parallel, e.g., in the text-picture combinations of illustrated documents. It is an important goal of this research not simply to merge the verbalization results of a natural language generator and the visualization results of a knowledge-based graphics generator, but to carefully coordinate graphics and text in such a way that they complement each other. We describe the architecture of the knowledge-based presentation system WIP* which guarantees a design process with a large degree of freedom that can be used to tailor the presentation to suit the specific context. In WIP, decisions of the language generator may influence graphics generation and graphical constraints may sometimes force decisions in the language production process, In this paper, we focus on the influence of graphical constraints on text generation. In particular, we describe the generation of cross-modal references, the revision of text due to graphical constraints and the clarification of graphics through text. particular combination of communication modes, the automatic generation of multimodal presentations is one of the tasks of such presentation systems. The task of the knowledge-based presentation system WIP is the generation of a variety of multimodal documents from an input consisting of a formal description of the communicative intent of a planned presentation. The generation process is controlled by a set of generation parameters such as target audience, presentation objective, resource limitations, and target language. One of the basic principles underlying the WIP project is that the various constituents of a multimodal presentation should be generated from a common representation. This raises the question of how to divide a given communicative goal into subgoals to be realized by the various mode-specific generators, so that they complement each other. To address this problem, we have to explore computational models of the cognitive decision processes coping with questions such as what should go into text, what should go into graphics, and which kinds of links between the verbal and nonverbal fragments are necessary.
منابع مشابه
Designing Illustrated Texts:
Multimodal interfaces combining, e.g., natural language and graphics take advantage of both the individual strength of each communication mode and the fact that several modes can be employed in parallel, e.g., in the text-picture combinations of illustrated documents. It is an important goal of this research not simply to merge the verbalization results of a natural language generator and the v...
متن کاملGeneration of texts for information graphics
We describe SelTex a text generation system for producing short texts and captions to accompany information graphics that are generated according to the writer’s intentions. SelTex uses rules that were extracted from a corpus study of more than 400 text excerpts. This corpus study shows that text and graphics play complementary roles in transmitting information from the writer to the reader. We...
متن کاملRedefining conceptions of grammar in English education in Asia: SFL in practice
This case study analyzes how a Taiwanese EFL teacher participating in a U.S. based MATESOL program made sense of systemic functional linguistics (SFL) and genre based pedagogy in designing and reflecting on literacy instruction for EFL learners in Taiwan. Using longitudinal ethnographic methods, the findings indicate that this teacher’s conceptualization of g...
متن کاملGeneration of Informative Texts with Style
An approach to the computational treatment of style is presented in the case of generation of informative texts. We regard the style mestly as a me,as of controlled selection of alternatives faced at each level of text generation. The generation technique, as well as the style specification, are considered at four levels-content production, discourse generation, surface structure development, a...
متن کاملThe production of documents from ontologies
The production of documents from an ontology is a challenging task which requires a significant effort from a natural language generator. Addressing this problem involves a careful examination of how the knowledge formalized in an ontology can be verbalized and realized. We have started to exploit the abilities of generating natural language texts from a Web Ontology Language (OWL) and to exami...
متن کامل